Abstract Title Page 

Not included in page count. 


Title: 

Which interventions have the greatest effect on student learning in Sub-Saharan Africa? A meta- 
analysis of rigorous impact evaluations 


Authors and Affiliations: 

Katharine Conn 

PhD candidate, Economics and Education, 
Columbia University, Teachers College 


SREE Spring 2014 Conference Abstract Template 



Abstract Body 

Limit 4 pages single-spaced. 


Background / Context: 

Description of prior research and its intellectual context. 

In the last three decades, there has been a large increase in the number of rigorous 
experimental and quasi-experimental evaluations of education programs in developing countries. 
These impact evaluations have taken place all over the globe, including a large number in Sub- 
Saharan Africa (SSA). For example, Blimpo and Evans (2011) examine the impact of school- 
based management training on subsequent student achievement in The Gambia, Piper and Korda 
(201 1) estimate the impact of adaptive instruction & teacher coaching on student learning in 
Liberia, and Kremer, Miguel and Thornton (2009) examine the impact of merit scholarships on 
student achievement in Kenya. 

Given the large number of programs in SSA, an important question is “what works” in 
terms of improving learning outcomes. Previous reviews have attempted to synthesize the large 
body of literature in developing countries, though not specific to SSA. Both Kremer and Holla 
(2009) and Kremer, Brannen, and Glennerster (2013) reviewed a selection of randomized 
controlled trials in the developing world and found that technology-assisted learning, remedial 
education, student tracking and the use of contract teachers were among the most effective 
interventions at improving student achievement. They note that technology-assisted learning and 
remedial education are particularly effective when the intervention is adaptive to the student’s 
learning level but that inputs such as hiring additional teachers, buying more textbooks, or 
providing flexible school grants have surprisingly small effects. Further, McEwan (2013) 
conducted a meta- analysis of 76 randomized experiments from the developing world and found 
that intervention types with the largest average effect sizes were those that employed computer- 
assisted learning, teacher training, and smaller classes/smaller learning groups/ ability grouping. 
Intervention types that were least effective included school monetary grants, and nutritional/ 
health treatments. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

These previous syntheses, however, have focused broadly on the developing world, not 
specifically on interventions and programs in SSA. The fact that the developing world is socially 
and economically diverse and that different interventions and programs are often culturally 
specific suggests that the broad results from previous syntheses may not be applicable to SSA, 
which is itself a quite diverse region. In this paper, I conduct a large systematic review of the 
impact evaluation literature throughout the region. This includes both experimental and quasi- 
experimental studies from all fields which evaluate the impact of education programs on student 
learning (fields include economics, health, psychology, and education). 

The purpose of this project is to better understand both which programs have been 
implemented and evaluated in the region and, of these, to determine which have the most 
promise in terms of increasing student learning outcomes in SSA - a critical policy imperative 
for developing countries with limited resources to spend. For example, do financial incentives 
for teachers result in similar outcomes as providing incentives to students or their families? Or 
are interventions that provide school supplies to students (textbooks, flipcharts) as effective as 
those that change the method of instruction? This paper focuses on analyzing this new body of 
rigorous education research in the context of SSA. 
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I also examine whether program effects differ by population subgroup (gender, income 
level etc.), region, education level (primary or secondary), subject matter (math, language, etc.), 
assessment type (researcher designed or standardized), target of intervention (student, classroom, 
teacher, or community), or intervention dosage (length or frequency of the intervention), among 
others. Lastly, I explore whether there are heterogeneous effects by study methodology (i.e., 
whether effects sizes are correlated with the type of experimental/ quasi-experimental strategy), 
study publication type, or study quality (quality index defined in full paper). 

Setting: 

Description of the research location. 

In this paper, I focus on educational interventions aimed at improving learning outcomes 
in SSA. This region includes East, Central, Southern, and Western Africa, as well as the Indian 
Ocean Islands. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

The study population for this meta-analysis includes evaluations of education programs at 
the primary and secondary level. I include interventions aimed at all students and at particular 
subgroups of students (e.g., single-sex education). Additionally, I focus only on studies that have 
taken place since 1980. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

The focus of this paper is to better understand the effects of interventions aimed at 
improving learning outcomes. This includes over thirteen different intervention types, ranging 
from pedagogical interventions (i.e. cooperative learning strategies or bilingual medium of 
instruction), to interventions in school health, to programs that alter student or teacher incentives. 
Table 1 includes a full list and explanation of program types (please insert Table 1 here). 

Within each intervention area, programs differ by treatment amount (treatments have 
different intensities or run for different amounts of time), the implementing agents of the 
program (NGOs, ministry of education, outside researchers), or by the specific sub-population 
targeted. Additionally, in this paper I focus only on experimental and quasi-experimental studies. 
This includes both large-scale, multi-site evaluations and single-site evaluations, as well as 
studies using propensity score matching, regression discontinuity, difference-in-differences, or 
instrumental variables designs. I exclude correlational studies. 

Research Design: 

Description of the research design. 

For my research design, I employ a random effects meta-analysis model (Lipsey and 
Wilson, 2001). I chose a random effects model (as opposed to a fixed effects model) because the 
interventions under study are relatively disparate in their program characteristics, even within an 
intervention area (for example, within school health interventions, I examine both iodine 
provision as well as de-worming programs). Thus I employ a random-effects model which 
allows me to estimate mean effect sizes, as opposed to a “common” or “true” effect sizes (used 
with fixed effects models). 
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Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

In order to better understand the literature, the first step of this systematic review was to 
conduct a large literature search. During this stage, careful attention was paid to the specification 
of inclusion/exclusion criteria as well as search terms. Efforts were made to include the gray 
literature - i.e., unpublished studies, such as those found through the World Bank and the MIT 
Poverty Action Lab, as well as other organizational publications. 

This literature search resulted in over 10,660 articles. The inclusion/exclusion criteria 
were set based upon the population, setting, and interventions mentioned above. After reading 
abstracts and papers, the final analyses includes 60 primary studies (within which are 75 
experiments). Table 2 indicates the process through which these 60 studies were selected (please 
insert Table 2 here). Importantly, one finding is that the number of studies taking place in SSA is 
much larger than that of previous reviews, which contained between 8-19 studies each (Kremer 
and Holla, 2009; Kremer, Brannen, and Glennerster, 2013; and McEwan, 2013). 

The final set of studies was coded based on an extensive set of questions regarding the 
population, setting, intervention, and study design. In order to reduce researcher-bias, a random 
subset was coded by a second reviewer; the inter-rater reliability was just over 98%. Next, the 
outcomes were converted into effect sizes. In this paper, I focus on Cohen’s d, the standardized 
mean difference, since this is the effect size commonly used in experiments. In many studies, 
particularly those evaluating pedagogical interventions, this involved correcting outcomes for 
clustering (Hedges, 2007). In order to combine and analyze these results, I then employ a random 
effects meta- analysis model (described above). Since many studies report multiple outcomes 
(without information on the correlation structure), in some analyses I use robust variance 
estimation (Hedges, Tipton, and Johnson, 2010), and when the number of studies is small, I use 
small-sample corrections (Tipton, in press). Finally, in addition to an overall analysis, I also 
conduct a “mini-meta-analysis” for each education topic (where data permits). 

Findings / Results: 

Description of the main findings with specific details. 

Overall effect size : The overall pooled effect size across all interventions is 0.16 
(SE=0.031; p = 0.0001). Importantly, the proportion of the observed variance in effect size 
estimates attributable to real difference across studies (I“ statistic) is 80.1% (x = 0.024). This I 
value is quite high (and relatively rare), but not unexpected given that I am pooling evaluations 
of studies across many intervention types. The forest plot in Figure 1 displays the full set of 
effect sizes under study (please insert Figure 1 here). 

Effect sizes by intervention type : Among those intervention types with the lowest pooled 
effect sizes are the abolishment of school fees (Cohen’s d = 0.007, p = 0.484), school-based 
management interventions (d= 0.047, p = 0.278), and school supplies provision (d=0.058, p = 
0.004). Interventions with the strongest the impact are comprehensive equity interventions 
(interventions that provide multiple inputs such as school infrastructure improvements, school 
meals, textbooks, and community programs as one package) [d = 0.38, p = 0.000]; instructional 
time increases (i.e. interventions that extend the school day) [d = 0.451, p = 0.000], and 
pedagogical interventions (i.e. interventions that implement new instructional methods) 

[d=1.128, p = 0.000]. The forest plot in Figure 2 displays the pooled effect sizes by intervention 
type (please insert Figure 2 here). 

Sub-group/moderator effects : While I conduct numerous sub-group/moderator analyses 
in my full paper, due to space constraints, I will briefly discuss my results for only two analyses 
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here: 1.) Pedagogical interventions have an average effect size 0.862 standard deviations higher 
than any other intervention type (SE= 0.105, p-value = 0.0001). Further, I find that pedagogical 
interventions that focus on procedural learning (executing actions sequences to solve problems) 
or conceptual learning (learning through classification/from examples) have the highest pooled 
effect sizes of 1.830 (p = 0.000) and 2.075 (p = 0.008) standard deviations, respectively. Inquiry- 
based learning interventions with a pooled effect size of 0.257 (p = 0.330) are the least effective 
of the pedagogical interventions, but even these programs seem to have a reasonable impact 
when compared to all non-pedagogical interventions. The forest plot in Figure 3 displays the 
results of this “mini -meta- analysis.” (please insert Figure 3 here). 

2.) In my second sub-group analysis, I examine how the impact of an intervention may 
vary depending on the “target” of the intervention (i.e. whether the subject of the intervention 
was the student, classroom, teacher, or community). I find that education interventions that target 
the classroom (i.e. tracking, class size, or instructional interventions) have the highest average 
effect size followed closely by those that target students (i.e. merit scholarships, or provision of 
uniforms, and textbooks), then followed by those that target teachers (i.e. teacher monetary 
incentives or contractual teachers). Interventions that target the entire community (provision of 
school report cards to the community or school management committee trainings) are estimated 
to have the lowest average impact. See the forest plot in Figure 4 for the various effect size 
magnitudes (please insert Figure 4 here). Overall, I find that interventions that target students or 
the entire classroom to be 1.13 standard deviations (SE= 0.254, p = 0.001) higher than those that 
target the teacher or school community. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

Overall, this meta-analysis finds that interventions in pedagogical methods (particularly 
those that emphasize procedural or conceptual learning) to be very promising. The high- 
magnitude effect sizes of these studies stand in contrast to interventions that have abolished 
school fees or improved school-based management (those with the lowest impact). This 
underscores the fact that interventions that may improve student enrollment or attendance (such 
as free/subsidized admission or community involvement) do not necessarily result in increased 
test scores; while children may attend school more frequently, they are not necessarily learning if 
school quality is low. However, evaluations of these high-impact pedagogical interventions are 
often quite small (randomization units under 30), and thus may suffer from various biases. 
Further, while other interventions that target the classroom (class size or student tracking) also 
seem to be promising, they are few in number. 

In general, while there is more rigorous evidence on interventions in education than 
previously reported, this evidence is not evenly distributed across topics or countries. In fact, 
when performing my literature search, I found that certain topics that are very relevant to the 
African context (such as multi-grade teaching or multi-shift teaching) are not under rigorous 
study at all. Further, over 30% of the studies in this meta-analysis come from Kenya alone. 

These findings caution against a strict interpretation of these results, as program effects may 
differ by country as well within country, depending on the initial conditions and binding 
constraints of different educational systems. [Note: In my full paper, I do test for contextual 
differences in program impact] . Recommendations for researchers would thus include the use of 
larger randomized trials, particularly in the field of pedagogical research, and an increased focus 
on evaluations that are more representative of both the continent and the issues facing education 
systems across Africa. 
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Not included in page count. 
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Appendix B. Tables and Figures 

Not included in page count. 


Table 1 


Intervention Area: 

Impact Evaluations examine: 

Abolishment of School 
Fees 

Universalization of schooling/ elimination of fees 

Class Size 

Class size reduction/increase 

Comprehensive Equity 
Program 

Programs which address inequalities in education through a myriad 
of strategies (for example, a combination of teacher training, 
supplemental education & increased instructional time) 

Cash Transfers 

Programs which offer monetary support to families, (can be 
conditional on their children’s school enrollment/ 
attendance/performance) 

Information for 
Accountability 

Interventions that provide student/ school performance information 
to parents/ communities in order to increase transparency & 
accountability. 

Instructional Time 

Increase/decrease in the number of hours that students attend 
school (can be altered through the length of a school day, length of 
the school year, multi-grade teaching, multi-shift teaching). 

Pedagogical 

Interventions 

Programs that affect the method(s) of instruction & learning; 
Changes in language of instruction policies; ICT computer 
programs employed both in and out of the classroom. 

School-Based 

Management 

Devolution of certain powers/responsibilities regarding education 
provision to the local level (parents/teachers/school community) 

School Health 

Programs which include both nutritional (school meals) and other 
health-related interventions (vitamins, de-worming drugs etc.) 

School Supplies 
Provision 

Provision of flipcharts, textbooks, writing materials etc. 

Student Incentives 

The provision of scholarships (or other incentives such as gifts) to 
students in order to incentivize sustained enrollment or improved 
test scores. 

Teacher Incentives 

Teacher or school-wide bonus pay (based on student achievement, 
as well as other select variables); Provision of temporary teacher 
contracts. 

Teacher Training 

Training of teachers in both pedagogical methods as well as 
academic content areas. 

Tracking & Peer 
Effects 

Grouping of students by ability level; influence of students from 
certain backgrounds/ability levels on rest of student population. 
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Table 2: 


Total relevant articles after following the search process above 
( + citation tracking etc.) 

10,660 

Total articles meeting criteria (based on abstract alone) 

168 

After a more thorough read of these 168 studies: 

> No. w/o a rigorous methodology 

-24 

> No. w/o achievement outcomes 

-29 

> No. outside of SSA 

-2 

> No. w/o explict findings 

-6 

> No. duplicates (earlier dates) 

-16 

> No. w/ non-school-aged population 

-3 

> No. whose findings can’t be standardized 

-28 


Total number of studies to be included in learning outcomes meta- analysis 60 
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Figure 1: 


The Impact of Various Education Interventions in Africa 


Random Effects Estimation ( page 1 of 3) 


Study 

ID 

ES (95% Cl) 

Class Size 


Duflo et al (2012), Kenya (Composite) 

► 0.23 (0.07, 0.38) 

Duflo et al (2012), Kenya (Language) 

► 0.23 (0.07, 0.38) 

Duflo et al (2012), Kenya (Math) 

i 0.19(0.03,0.35) 

Tracking & Peer Effects 


Duflo et al (201 1 ), Kenya (Composite) 

f 0.18(0.03,0.33) 

Duflo et al (201 1 ), Kenya (Language) 

i 0.17 (-0.03, 0.36) 

Duflo et al (201 1 ), Kenya (Math) 

' 0.16 (-0.01, 0.32) 

Pedagogical Intervention 


Plomp et al (1991), Botswana (Math) 

*- 0.39 (-0.90, 1.68) 

Piper & Korda (2011), Liberia (Composite) 

• 0.46 (0.25, 0.67) 

Adeleke (2007), Nigeria (a) (Math) 

2.45(1.22,3.69) 

Adeleke (2007), Nigeria (b) (Math) 

* 3.65 (2.34,4.95) 

Agbatogun (2012), Nigeria (Language) 

1.41 (-0.17, 2.98) 

Nwagbo (2006), Nigeria (Biology) * 

f 0.04 (-0.61, 0.68) 

Akinsola & Animasahun (2007), Nigeria (Math) 

•- 0.53 (-0.61, 1.66) 

Ajaja & Eravwoke (2010), Nigeria (Science) 

♦ 1.95(0.94,2.96) 

Sarfo & Elen (2007), Ghana (Technical problem-solving) 

•- 0.85 (-0.50, 2.20) 

Wachanga & Mwangi (2004), Kenya (Chemistry) 

• 0.46 (-0.17, 1.09) 

Van Staden (2011), South Africa (Language) 

♦ 1.74 (1.47, 2.01) 

Githau & Nyabwa (2008), Kenya (Math) 

0.59 (-0.26, 1.44) 

Onu et al (2012), Nigeria (Math) 

*- 0.98 (-0.70, 2.66) 

Kolawole (2008), Nigeria (Math) 

♦ 1.21(0.41,2.00) 

Comprehensive Equity Program 


Levy et al (2009), Burkina Faso (Composite) 

► 0.38 (0.29, 0.47) 

Levy et al (2009), Burkina Faso (Language) 

► 0.37 (0.28, 0.45) 

Levy et al (2009), Burkina Faso (Math) 

► 0.40 (0.30, 0.49) 

Instructional Time 


Orkin (2013), Ethiopia (Composite) 

• 0.45(0.11,0.80) 

Orkin (2013), Ethiopia (Language) 

• 0.49(0.15,0.83) 

Orkin (2013), Ethiopia (Math) 

► 0.41 (0.07, 0.76) 

1 

1 


-4.95 0 4.95 
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Figure 1 (continued): 


The Impact of Various Education Interventions in Africa 

Random Effects Estimation (page 2 of 3) 


Study 

ID 

ES (95% Cl) 

Teacher Incentives 


Duflo et al (2012), Kenya (Composite) 

♦ 0.23(0.11,0.34) 

Duflo et ai (2012), Kenya (Language) 

♦ 0.18(0.04,0.31) 

Duflo et al (2012), Kenya (Math) 

♦ 0.26(0.14,0.38) 

Glewwe et al (2010), Kenya (a) (Composite) 

0.14 (-0.00, 0.28) 

Glewwe et al (2010), Kenya (b) (Composite) * 

-0.02 (-0.14, 0.11) 

School-Based Management 


Duflo et al (2012), Kenya (Composite) 

*■ 0.13 (-0.08, 0.34) 

Duflo et ai (2012), Kenya (Language) 

►- 0.05 (-0.17, 0.28) 

Duflo et al (2012), Kenya (Math) 

0.19(0.02,0.35) 

Barr et al (2012), Uganda (b) (Composite) 

*- 0.11 (-0.13, 0.34) 

Barr et al (2012), Uganda (a) (Composite) 

0.22 (0.00, 0.44) 


Blimbo & Evans (201 1 ), The Gambia (Composite) 
Blimbo & Evans (201 1 ), The Gambia (Language) 
Blimbo & Evans (201 1 ), The Gambia (Math) 
Glewwe & Mai'ga (2011), Madagascar (Composite) 


-0.08 (-0.25, 0.09) 
-0.04 (-0.22, 0.14) 
-0.12 (-0.28, 0.04) 
0.07 (-0.19, 0.33) 


Information for Accountability 
Bjorkman (2006), Uganda (Composite) 
Nguyen (2008), Madagascar (a) (Composite) 
Nguyen (2008), Madagascar (b) (Composite) 
Piper & Korda (2011), Liberia (Composite) 
Piper & Korda (2011), Liberia (Language) 
Piper & Korda (2011), Liberia (Math) 


0.33 (-0.02, 0.68) 
0.24 (0.02, 0.45) 
0.08 (-0.08, 0.24) 
0.05 (-0.16, 0.26) 
0.05 (-0.16, 0.26) 
0.05 (-0.16, 0.26) 


Student Incentives 

Blimpo (2010), Benin (a) (Composite) 
Blimpo (2010), Benin (b) (Composite) 
Blimpo (2010), Benin (c.) (Composite) 
Evans et al (2009), Kenya (Composite) 
Kremer et al (2009), Kenya (Composite) 


0.29 (0.05, 0.53) 
0.34 (0.09, 0.59) 
0.27 (-0.02, 0.56) 
0.25 (-0.08, 0.58) 
0.27 (-0.04, 0.58) 


School Supplies Provision 

□as et al (2011), Zambia (Composite) 

Das et al (2011), Zambia (Language) 

Das et al (2011), Zambia (Math) 

Glewwe et al (2004), Kenya (Composite) 

Glewwe et al (2008), Kenya (Composite) 

Frolich & Michaelowa (2011), 5 Francophone countries (Composite) 
Frolich & Michaelowa (2011), 5 Francophone countries (Language) 
Frolich & Michaelowa (2011), 5 Francophone countries (Math) 


0 . 10 ( 0 . 00 , 0 . 20 ) 
0 . 10 ( 0 . 00 , 0 . 20 ) 
0.10(0.00, 0.19) 
0.01 (-0.05, 0.07) 
0.02 (-0.15, 0.19) 
0.20 (-0.27, 0.66) 
0.26 (-0.29, 0.82) 
0.13 (-0.38, 0.64) 


-.819 0 .819 
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Figure 1 (continued): 


The Impact of Various Education Interventions in Africa* 

Random Effects Estimation (page 3 of 3) 


Study 

ID ES (95% Cl) 


Cash Transfers 

Baird et al (201 1 ), Malawi (a) (Cognition) 

Baird et al (201 1 ), Malawi (a) (Composite) 

Baird et al (201 1 ), Malawi (a) (Language) 

Baird et al (201 1 ), Malawi (a) (Math) 

Baird et al (201 1 ), Malawi (a) (Language) 

Baird et al (201 1 ), Malawi (b) (Cognition) 

Baird et al (201 1 ), Malawi (b) (Composite) 

Baird et al (201 1 ), Malawi (b) (Language) 

Baird et al (201 1 ), Malawi (b) (Math) 

Baird et al (201 1 ), Malawi (b) (Language) 

DSD, SASSA & UNICEF (2012), South Africa (Math) 


k 0.17(0.08,0.27) 

• 0.12(0.01,0.24) 

0.14(0.03, 0.25) 

I 0.10 (- 0 . 02 , 0 . 22 ) 

0.02 (-0.16, 0.19) 

0.14 (-0.10, 0.37) 

0.00 (-0.17, 0.17) 

-0.03 (-0.19,0.13) 
0.03 (-0.15, 0.22) 
0.25 (0.03, 0.46) 
f 0.09 (-0.07, 0.24) 


School Health 

Whaley et al (2003), Kenya (a) (Cognition) 

Whaley et al (2003), Kenya (a) (Composite) 

Whaley et al (2003), Kenya (a) (Language) 

Whaley et al (2003), Kenya (a) (Math) 

Whaley et al (2003), Kenya (b) (Cognition) 

Whaley et al (2003), Kenya (b) (Composite) 

Whaley et al (2003), Kenya (b) (Language) 

Whaley et al (2003), Kenya (b) (Math) 

Whaley et al (2003), Kenya (c.) (Cognition) 

Whaley et al (2003), Kenya (c.) (Composite) 

Whaley et al (2003), Kenya (c.) (Language) 

Whaley et al (2003), Kenya (c.) (Math) 

Miguel & Kremer (2004), Kenya (Composite) 

Vermeersch & Kremer (2004), Kenya (Cognition) 

Vermeersch & Kremer (2004), Kenya (Composite) 

Vermeersch & Kremer (2004), Kenya (Composite-Oral Curricular) 
Vermeersch & Kremer (2004), Kenya (Composite-Written Curriculuar) 
Jukes et al (2006), The Gambia (Composite) 

Gee (2010), Kenya (Cognition) 

Gee (2010), Kenya (Cognition) 

Gee (2010), Kenya (Composite) 

Clarke et al (2008), Kenya (Cognition-Composite) 

Clarke et al (2008), Kenya (Cognition-Language) 

Clarke et al (2008), Kenya (Cognition-Math) 

Clarke et al (2008), Kenya (Social Science) 

van Stuijvenberg (1999), South Africa (Cognition-Composite) 

Baumgartner et al (2012), South Africa (Cognition) 

Baumgartner et al (2012), South Africa (Cognition) 

Baumgartner et al (2012), South Africa (Composite) 

Baumgartner et al (2012), South Africa (Cognition) 

Baumgartner et al (2012), South Africa (Cognition) 

Baumgartner et al (2012), South Africa (Composite) 

Baumgartner et al (2012), South Africa (Cognition) 

Baumgartner et al (2012), South Africa (Cognition) 

Baumgartner et al (2012), South Africa (Composite) 

Abolishment of school fees 

Lucas & Mbiti (2012), Kenya (Composite) 




0.21 (-0.66, 1.07) 
0.18 (-0.69, 1.05) 
0.11 (-0.76, 0.98) 
0.23 (-0.64, 1.09) 
-0.20 (-1.06,0.65) 
-0.05 (-0.90, 0.80) 
0.03 (-0.82, 0.88) 
0.04 (-0.81,0.89) 
-0.04 (-0.88, 0.80) 
0.15 (-0.70, 0.99) 
0.15 (-0.69, 1.00) 
0.32 (-0.52, 1.17) 
0.00 (-0.14, 0.14) 
-0.03 (-0.21,0.15) 
0.02 (-0.18, 0.22) 
0.07 (-0.11,0.25) 
0.02 (-0.22, 0.26) 
0.12 (-0.17, 0.40) 
0.19(0.06, 0.32) 
0.17 (-0.05, 0.39) 
0.18(0.01,0.35) 
0.22 (0.06, 0.37) 
0.17(0.02, 0.32) 
0.26 (0.11,0.42) 
0.02 (-0.18, 0.22) 
-0.01 (-0.35, 0.34) 
-0.12 (-0.43, 0.19) 
0.09 (-0.22, 0.39) 
-0.02 (-0.33, 0.29) 
-0.09 (-0.40, 0.22) 
-0.05 (-0.36, 0.26) 
-0.07 (-0.38, 0.24) 
-0.04 (-0.35, 0.28) 
-0.04 (-0.35, 0.27) 
-0.04 (-0.35, 0.27) 


0.01 (-0.01, 0.03) 


-1.17 0 1.17 


*NOTE: The purpose of this Forest Plot is to illustrate the effect sizes and variances used in this meta-analysis. It 
does not include a total effect size, or weights, since code currently does not exist for integrating forest plots into the 
RVE framework. See the text for the overall average effect size. 
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Figure 2: Forest plot for random effects meta-analysis, all intervention types. 


The Impact of Various Education Intervention Types in Africa 

Random Effects Estimation (results pooled by intervention type ) 


Study 

ID 


ES (95% Cl) 

% 

Weight 

Class Size 

Subtotal (l-squared = 0.0%, p = 0.928) 

1 

P 

0.22 (0.12, 0.31) 

4.04 

Teacher Incentives 

Subtotal (l-squared = 66.1%, p = 0.019) 

1 

P 

1 

0.16(0.06,0.26) 

7.26 

School-Based Management 
Subtotal (l-squared = 40.4%, p = 0.098) 

1 

1 

1 

0.05 (-0.04, 0.13) 

10.65 

Tracking & Peer Effects 

Subtotal (l-squared = 0.0%, p = 0.984) 

1 

P 

0.17(0.07, 0.26) 

3.91 

Cash Transfers 

Subtotal (l-squared = 0.0%, p = 0.456) 

1 

1 

t 

1 

0.11 (0.07, 0.15) 

14.76 

Information for Accountability 
Subtotal (l-squared = 0.0%, p = 0.566) 

1 

1 

9 

0.10 (0.02, 0.19) 

6.66 

Student Incentives 

Subtotal (l-squared = 0.0%, p = 0.994) 

1 

0 

i 

0.29 (0.16, 0.41) 

4.54 

School Health 

Subtotal (l-squared = 0.0%, p = 0.842) 

i 

i 

9 

0.09 (0.05, 0.13) 

26.76 

School Supplies Provision 
Subtotal (l-squared = 0.0%, p = 0.553) 

l 

i! 

I 

0.06 (0.02, 0.10) 

8.93 

Pedagogical Intervention 

Subtotal (l-squared = 85.7%, p = 0.000) 

1 o 

1.13(0.64, 1.61) 

3.85 

Comprehensive Equity Program 
Subtotal (l-squared = 0.0%, p = 0.905) 

i 

i 

;o 

0.38 (0.33, 0.43) 

4.70 

Instructional Time 

Subtotal (l-squared = 0.0%, p = 0.952) 

io 

1 

0.45 (0.25, 0.65) 

2.23 

Abolishment of school fees 
Subtotal (l-squared = .%, p = .) 

1 

1 

1 

1 

0.01 (-0.01,0.03) 

1.70 

Overall (l-squared = 80.1%, p = 0.000) 

i 

0.16 (0.12, 0.20) 

100.00 

NOTE: Weiahts are from random effects anal\ 

iis 



1 

-1.61 

i 

1.61 
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Figure 3: Forest plot for random effects meta- analysis, pedagogical interventions only. 


The Impact of Various Pedagogical Interventions 

Random Effects Estimation 


Study 

ID 



ES (95% Cl) 

% 

Weight 

inquiry-based learning 

Nwagbo (2006), Nigeria 

- 

1 

1 

h' 

0.04 (-0.61, 0.68) 

8.68 

Akinsola & Animasahun (2007), Nigeria 


♦t 

0.53 (-0.61, 1.66) 

6.55 

Sarfo & Elen (2007), Ghana 



0.85 (-0.50, 2.20) 

5.70 

Subtotal (l-squared = 0.0%, p = 0.496) 

' 

>: 

0.26 (-0.26, 0.77) 

20.93 

procedural learning 

Van Staden (201 1 ), South Africa 
Adeleke (2007), Nigeria (a) 


i 

1.74 (1.47, 2.01) 
2.45 (1.22, 3.69) 

9.93 

6.14 

Subtotal (l-squared = 17.6%, p = 0.271) 


lO 

1 

1.83 (1.37, 2.29) 

16.07 

metacognitive strategies 

Onu et al (2012), Nigeria 
Subtotal (l-squared = .%, p = .) 

< 

1 

1 

2> 

0.98 (-0.70, 2.66) 
0.98 (-0.70, 2.66) 

4.58 

4.58 

computer-assisted learning 

Plomp et al (1991), Botswana 


1 

1 

0.39 (-0.90, 1.68) 

5.94 

Agbatogun (2012), Nigeria 



1.41 (-0.17, 2.98) 

4.92 

Subtotal (l-squared = 0.0%, p = 0.327) 

i 

i 

0.80 (-0.20, 1.79) 

10.86 

conceptual learning 

Adeleke (2007), Nigeria (b) 


i 

i 

3.65 (2.34, 4.95) 

5.87 

Githau & Nyabwa (2008), Kenya 
Subtotal (l-squared = 93.2%, p = 0.000) 

cooperative learning 


4 

0.59 (-0.26, 1.44) 
2.07 (-0.92, 5.07) 

7.79 

13.66 


i 

i 

i 

Ajaja & Eravwoke (2010), Nigeria 



1.95 (0.94, 2.96) 

7.09 

Wachanga & Mwangi (2004), Kenya 


A 

0.46 (-0.17, 1.09) 

8.72 

Kolawole (2008), Nigeria 



1.21 (0.41, 2.00) 

8.03 

Subtotal (l-squared = 69.0%, p = 0.040) 


<i> 

i 

1.13 (0.30, 1.96) 

23.84 

adaptive instruction & teacher coaching 

Piper & Korda (2011), Liberia 


i 

i 

♦i 

0.46 (0.25, 0.67) 

10.06 

Subtotal (l-squared = .%, p = .) 


o! 

0.46 (0.25, 0.67) 

10.06 

Overall (l-squared = 85.7%, p = 0.000) 


<?> 

1.13 (0.64, 1.61) 

100.00 

NOTE: Weiahts are from random effects analysis 

r 




-5.07 0 5.07 
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Figure 4: Forest plot for random effects meta-analysis (sub-group = intervention target) 


How does achievement impact vary with the target of the intervention? 

Random Effects Estimation (pooled by sub-group) 


Study 

ID 


ES (95% Cl) 

% 

Weight 

Intervention Target = Teacher 

Subtotal (l-squared = 66.1%, p = 0.019) 

4 

I 

0.16 (0.06, 0.26) 

13.03 

Intervention Target = Class 

Subtotal (l-squared = 87.6%, p = 0.000) 

i 

l 

l 

1 

iO 0.38 (0.26, 0.50) 
1 

44.51 

Intervention Target = Community 

Subtotal (l-squared = 25.3%, p = 0.175) 

oi 

1 

0.07 (0.01, 0.13) 

33.19 

Intervention Target = Student 

Subtotal (l-squared = 0.0%, p = 0.994) 

l 

1 

0.29 (0.16, 0.41) 

9.27 

Overall (l-squared = 80.9%, p = 0.000) 


0.22 (0.16, 0.29) 

100.00 

NOTE: Weights are from random effects an. 

ilysis 




-.497 0 


.497 
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